Problems of decision rule elicitation in a classification task

Intelligent decision support requires knowledge elicitation processes. Two primary approaches for knowledge elicitation in a multiattribute classification task are 1) direct elicitation of decision rules in the form of productions, and 2) classification of multiattribute objects by an expert as a basis for development of the underlying decision rules. This study reports an experiment using a simple classification task, to compare these two forms of knowledge elicitation. Relative consistency and complexity of the resulting rule bases are analyzed. System CLASS was used as a tool for the second approach, as well as a means of analysis for the first approach. It was found that it was easier for subjects to accomplish the task using object classification than it was to formulate production rules directly. High degrees of inconsistency and incomplete rule bases resulted when there was no computer aid for the process of knowledge elicitation.